A memristor, also sometimes referred to as a “memory resistor” is a passive two-terminal electrical component that can function as a non-linear circuit element relating charge and magnetic flux linkage. When current flows in one direction through the device, the electrical resistance can increase. When current flows in the opposite direction, the resistance can decrease. When the current is stopped, the device retains the last resistance that it had, and when the flow of charge starts again, the resistance of the circuit will be what it was when it was last active. A memristor thus has a regime of operation with an approximately linear charge-resistance relationship as long as the time-integral of the current stays within certain bounds.
Devices based on thermodynamic computing have been implemented. One example of such a device is the Knowm™ network or system, which is described in a number of U.S. patents and publications publications. U.S. Pat. No. 6,889,216, entitled “Physical Neural Network Design Incorporating Nanotechnology,” which issued to Alex Nugent on May 3, 2005 generally describes a physical neural network that generally includes one or more neuron-like nodes, which are formed from a plurality of interconnected nanoconnections formed from nanoconductors. Such connections constitute Knowm™ connections. Each neuron-like node sums one or more input signals and generates one or more output signals based on a threshold associated with the input signal.
The Knowm™ device physical neural network also includes a connection network formed from the interconnected nanoconnections, such that the interconnected nanoconnections used thereof by one or more of the neuron-like nodes are strengthened or weakened according to an application of an electric field, variations in frequency, and so forth. U.S. Pat. No. 6,889,216 is incorporated herein by reference.
Another example of a Knowm™ network or system is described in U.S. Patent Publication No. 20030236760, entitled “Multi-layer Training in a Physical Neural Network Formed Utilizing Nanotechnology,” by inventor Alex Nugent, which was published on Dec. 25, 2003. U.S. Patent Publication No. 20030236760 generally describes methods and systems for training at least one connection network located between neuron layers within a multi-layer physical neural network (e.g., a Knowm™ network or device). The multi-layer physical neural network described in U.S. Patent Publication No. 20030236760 can be formed with a plurality of inputs and a plurality outputs thereof, wherein the multi-layer physical neural network comprises a plurality of layers therein, such that each layer thereof comprises at least one connection network and at least one associated neuron.
Thereafter, a training wave, as further described in U.S. Patent Publication No. 20030236760, can be initiated across one or more connection networks associated with an initial layer of the multi-layer physical neural network which propagates thereafter through succeeding connection networks of succeeding layers of the multi-layer physical neural network by successively closing and opening at least one switch associated with each layer of the multi-layer physical neural network. At least one feedback signal thereof can be automatically provided to each preceding connection network associated with each preceding layer thereof to strengthen or weaken nanoconnections associated with each connection network of the multi-layer physical neural network. U.S. Patent Publication No. 20030236760 is incorporated herein by reference.
A further example of a Knowm™ network or system is described in U.S. Patent Publication No. 20040039717, entitled High-density synapse chip using nanoparticles” by inventor Alex Nugent. U.S. Patent Publication No. 20040039717 published on Feb. 26, 2004 and generally describes a physical neural network synapse chip (i.e., a Knowm™ chip) and a method for forming such a synapse chip. The synapse or Knowm™ chip can be configured to include an input layer comprising a plurality of input electrodes and an output layer comprising a plurality of output electrodes, such that the output electrodes are located perpendicular to the input electrodes. A gap is generally formed between the input layer and the output layer.
A solution can then be provided which is prepared from a plurality of nanoconductors and a dielectric solvent. The solution is located within the gap, such that an electric field is applied across the gap from the input layer to the output layer to form nanoconnections of a physical neural network implemented by the synapse chip. Such a gap can thus be configured as an electrode gap. The input electrodes can be configured as an array of input electrodes, while the output electrodes can be configured as an array of output electrodes. U.S. Patent Publication No. 20040039717 is also incorporated herein by reference.
A further example of a Knowm™ network or system is disclosed in U.S. Patent Publication No. 20040153426, entitled “Physical Neural Network Liquid State Machine Utilizing Nanotechnology,” by inventor Alex Nugent, which was published on Aug. 5, 2004. U.S. Patent Publication No. 20040153426 generally discloses a physical neural network (i.e., a Knowm™ network), which functions as a liquid state machine.
The physical neural network described in U.S. Patent Publication No. 20040153426 can be configured from molecular connections located within a dielectric solvent between pre-synaptic and post-synaptic electrodes thereof, such that the molecular connections are strengthened or weakened according to an application of an electric field or a frequency thereof to provide physical neural network connections thereof. A supervised learning mechanism is associated with the liquid state machine, whereby connections strengths of the molecular connections are determined by pre-synaptic and post-synaptic activity respectively associated with the pre-synaptic and post-synaptic electrodes, wherein the liquid state machine comprises a dynamic fading memory mechanism. U.S. Patent Publication No. 20040153426 is also incorporated herein by reference.
A further example of a Knowm™ network or system is disclosed in U.S. Patent Publication No. 20040162796, entitled “Application of Hebbian and anti-Hebbian Learning to Nanotechnology-based Physical Neural Networks” by inventor Alex Nugent, which published on Aug. 19, 2004. U.S. Patent Publication No. 20040162796 generally discloses a physical neural network (i.e., Knowm™ network) configured utilizing nanotechnology. The Knowm™ network disclosed in U.S. Patent Publication No. 20040162796 includes a plurality of molecular conductors (e.g., nanoconductors) which form neural connections between pre-synaptic and post-synaptic components of the physical neural network.
An alternative example of a nanoscale meta-stable switching element is the cross bar architecture. A molecular cross bar memory is disclosed, for example, in U.S. Pat. No. 6,128,214 entitled “Molecular Wire Cross bar Memory” which issued to Kuekes et al. on Oct. 3, 2000. Kuekes et al disclose a memory device that is constructed from cross bar arrays of nanowires sandwiching molecules that act as on/off switches. The device is formed from a plurality of nanometer-scale devices, each device comprising a junction formed by a pair of crossed wires where one wire crosses another and at least one connector species connects the pair of crossed wires in the junction. The connector species comprises a bi-stable molecular switch. The junction forms either a resistor or a diode or an asymmetric non-linear resistor. The junction possesses a state that is capable of being altered by application of a first voltage and sensed by the application of a second, non-destructive voltage. A series of related technology attempts to convert everything from molecular logic to how to chemically assemble these devices.
Such a molecular cross bar device has two general applications. The notion of transistors built from nanotubes and relying on nanotubes properties is being pursued. In this manner, computational systems can be constructed. Second, two wires can be selectively brought to a certain voltage and the resulting electrostatic force attracts them. When the wires touch, the Van der Waals force maintains the wires in contact with one another such that a “bit” is stored. The connections in such an apparatus can therefore be utilized with standard electronic circuitry.
One aspect of the cross bar architecture, which deserves some illumination, is its potential unreliability. The device functions by creating a physical system with one meta-stable state, which is formed from the balance of van der-walls intermolecular attraction and a mechanical tension from a bent nanowire. The van der-walls force must be made sufficient to oppose the mechanical strain. The construction of such a device preferably utilizes a nanowire suspended above a lower nanotube (or other nanoparticle) or electrical contact. The further apart the supports, the less of a force required to bend the nanowire. This has an interesting consequence when the system in scaled down.
As the switch density is increased, the support distance must necessarily decrease. This causes the force from mechanical strain to increase. One possible solution would be to place the nanowires closer together, so that less of a deflection is needed to make contact. There is an absolute distance, however, for which the nanowires cannot be brought closer, and this distance is set by quantum mechanical electron tunneling. In this manner, it can be seen that as the device is scaled down, the potential energy well formed from the Van der Waals force becomes weakened as the mechanical strain from the bent nanowire increases. This results in the connection having an increased probability of falling into the “ground state”.
Alternately, switching molecules have been found that can be configured to be in a conducting and a non-conducting state. To date, one problem with such molecules is that the states are only meta-stable, particularly one of the states. After a short time, which usually decreases as the temperature is increased, the state is lost. If the “on” state represents a conducting state, then this naturally raises the question, “how does one work with an unreliable connection?” It is the purpose of this disclosure to apply a type of local interaction so as to work with an unreliable connection, or alternately to constantly repair a connection if it undergoes a transition from a meta-stable to a ground state. This methodology can be used in, for example, Knowm™ connection networks, nanowire cross junctions and meta-stable switching molecules such as rotoxane. The methodology has been first described for Knowm™ connection networks, and its extension to other nanoscale implementations is unexpected. In all cases, we can refer to pre- and post-synaptic electrodes, where a Knowm™ connection, nanowire junction, switching molecule, quantum tunneling transistor and the like forms the pre-to-post electrode switching contact. In all cases, the device is configured to work with a meta-stable nanoscale switch, which we will describe in detail and offer examples.
A local feedback mechanism can be applied, which implements Hebbian and anti-Hebbian learning. Such a learning mechanism can utilize a voltage gradient or voltage gradient dependencies to implement Hebbian and/or anti-Hebbian (AHAH) plasticity. The learning mechanism can also utilize pre-synaptic and post-synaptic frequencies to provide Hebbian and/or anti-Hebbian learning within the physical neural network. U.S. Patent Publication No. 20040162796 is incorporated herein by reference.
An additional example of a Knowm™ network or device is disclosed in U.S. Patent Publication No. 20040193558, entitled “Adaptive Neural Network Utilizing Nanotechnology-based Components” by Alex Nugent, which published on Sep. 30, 2004. U.S. Patent Publication No. 20040193558 generally describes methods and systems for modifying at least one synapse of a physical neural network (i.e., a Knowm™ network). The physical neural or Knowm™ network described in U.S. Patent Publication No. 20040193558 can be implemented as an adaptive neural network, which includes one or more neurons and one or more synapses thereof.
The synapses are formed from a plurality of nanoparticles disposed within a dielectric solution in association with one or more pre-synaptic electrodes and one or more post-synaptic electrodes and an applied electric field. Alternately, the synapses are formed from a plurality of nanowire cross junctions formed from two or more nanowires composed of, for example, carbon nanotubes and forming one or more pre-synaptic electrodes and one or more post-synaptic electrodes. Alternately, The synapses are formed from a plurality of switching molecules formed from one or more molecule of, for example, Rotoxane, preferably located between two or more pre- and post-synaptic electrodes comprising, for example, carbon nanotubes or photolithography defined and patterned electrodes.
At least one electric pulse can be generated from one or more of the neural circuit modules to one or more of the pre-synaptic electrodes of a succeeding neuron and one or more post-synaptic electrodes of one or more of the neurons of the synapse junction, thereby strengthening at least one nanoparticle of a plurality of nanoparticles disposed within the dielectric solution and at least one synapse thereof. At least one electric pulse can be generated from one or more of the neural circuit modules to one or more of the pre-synaptic electrodes of a succeeding neuron and one or more post-synaptic electrodes of one or more of the neurons of the synapse junction, thereby configuring the state of at least one meta-stable switch of a plurality of meta-stable switches disposed within the dielectric solution and at least one synapse thereof. U.S. Patent Publication No. 20040193558 is incorporated herein by reference.
Another example of a Knowm™ network or device is disclosed U.S. Patent Publication No. 20050015351, entitled “Nanotechnology Neural Network Methods and Systems” by inventor Alex Nugent, which published on Jan. 20, 2005. U.S. Patent Publication No. 20050015351 generally discloses a physical neural network (i.e., a Knowm™ network), which constitutes a connection network comprising a plurality of molecular conducting connections suspended within a connection gap formed between one or more input electrodes and one or more output electrodes. One or more molecular connections of the molecular conducting connections can be strengthened or weakened according to an application of an electric field, frequency, and the like across the connection gap.
The notion of a connection network can be applied to a nanoscale architecture of the cross bar array. Rather than a Knowm™ synapse forming a connection at the intersection of the cross bar, the connection is formed by the mechanical contact of the electrodes themselves, which are implemented as nanowires. When two wires are raised to opposing voltages, columbic attraction brings the wires together. Charging the wires can be used to overcome the Van der Waals attractive force and allows the mechanical strain to bring the nanowire to its ground state. The same self-organizing principles that were developed for a Knowm connection can be applied to an unreliable cross-bar electrode junction.
The notion of a connection network can be applied to a nanoscale switching molecule architecture. Rather than a Knowm™ synapse forming a connection at the intersection of the cross bar, the connection is formed by one or more switching molecules composed of, but not limited to, Rotoxane. In general, all that is required is a two-state system, where an impedance change can be measured between the two states. In addition, it is preferable that one of the states is unstable and has a non-zero probability of transition to the ground state. When two wires are raised to opposing voltages, an electric field is generated that switches the molecules into the meta-stable state. The same self-organizing principles that were developed for a Knowm connection can be applied to a meta-stable switching molecule architecture.
An example of meta-stable switching is disclosed in U.S. Pat. No. 7,599,895 entitled, “Methodlogy for the Configuration and Repair of Unreliable Switching Elements,” which issued on Oct. 6, 2009 to Alex Nugent, which is incorporated herein by reference in its entirety.
As transistor densities on modern integrated electronic chips increase, there is a growing trend toward reconfigurable architectures. Rather than implementing application specific integrated circuits (ASIC), it is preferred that a design be deployed on programmable logic devices. The move in such a direction is creating a growing trend toward an IP-based development process, where circuits are defined by their programming routine rather than the actual physical layout. Rather than implementing a program to run on a processor, for example, a chip may process a program to build the processor.
In view of the foregoing developments in thermodynamic computing and the need for reconfigurable architectures it is believed that one solution toward creating such technology involves the implementation of improved memristor devices, systems and components.